Syntactic Folding and its Application to the Information Extraction from Web Pages

The paper deals with investigations concerning potential structures of documents that will be subject to automated information extraction. The focus is on folding principles and their influence on the recognition of certain data in a document undergoing the extraction. Introduction The topic of our work is information extraction from the Internet. There are a couple of approaches which deal with the problem of recognizing structural data in semistructured documents for retrieval of user specified information from these and from similar documents (possibly of the same source), in an automatic semi-antomatic way (Freitag 1996), (Soderland 1997), (Kushmerick 1997). Ideally, structural information shall be learned by presenting only samples of text segments which a user wants to extract from these pages to a learning device, without any need to specify details of how the desired samples can be localized within the document. The learning device should generate a procedure, a wrapper, that reading the same documents puts out a collection of information, including the samples and, hopefully, extending them in terms of finding similar items. These approaches led to a variety of wrapper classes, e.g. LR-wrappers (Kushmerick 2000), island wrappers (Grieser et al. 2000), T-wrappers (Thomas 1999) and further variants of them with different characteristics, for instance (Hsu & Dung 1998), (Muslea, Minton, Knoblock 1999). Extracting the fundamentals of these approaches, our goal consists in a specification of the area of application for a wrapper under investigation, and in finding out some rules for their conscious selection and generation. On the basis of (Grieser et al. 2000) and a comparison with Kushmerick’s wrapper classes, this publication will focus on the question how tupels, that shall be Copyright (~) 2001, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. extracted from a document in the manner above, can interrelate. It is not our intention to set tile mentioned wrapper concepts into a relation in terms of investigating the acceptable languages in detail. To restrict the domain, we make a couple of assumptions: The information, we are looking for, should be encapsulated by syntactic expressions ttlat are comparable in some manner. We suppose there are a lot of web pages which are organized in a similar way and which make use of the same expressions for structuring information. However, we do not require a specific set of such expressions, so the approach is not restricted to HTML-documents. HTML serves more for azl application domain.